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1.  Introduction 

Because  of  the  significant  computational  limitations  of  integer  pro- 
gramming algorithms  for  obtaining  an  optimal  solution,  there  has  been 
considerable  research  in  recent  years  on  heuristic  algorithms  for  efficiently 
Peeking  very  good  solutions  that  are  not  guaranteed  to  be  optimal.  For  ex- 
ample, such  algorithms  have  been  developed  by  Reiter  and  Rice  [10],  Echols 
and  Cooper  [1],  Senju  and  Toyoda  [12],  Hillier  [6,5],  Roth  [11],  Kochenberger 
McCarl,  and  Wyman  [8],  Ibaraki,  Ohashi,  and  Mine  [7],  and  Toyoda  [14].  Al- 
though their  primary  motivation  was  to  provide  an  effective  way  of  dealing 
with  the  frequently  encountered  Integer  programming  problems  that  are  beyond 
the  computational  capability  of  exact  algorithms,  heuristic  algorithms  also 
can  be  very  useful  on  smaller  problems  by  providing  an  advanced  starting  solu- 
tion to  accelerate  an  exact  algoritlim.  For  example,  the  availability  of  an 
advanced  starting  solution  has  been  exploited  very  advantageously  by  the 
authors  in  Hillier' s Bound-and-Scan  algorithm  [4]  and  Faaland  and  Hillier' s 
Accelerated  Bound-and-Scan  algorithm  [2]. 


Some  of  these  heuristic  algorithms  are  expressly  designed  for  binary 
(0-1)  Integer  programming  while  others  deal  with  the  general  integer  linear 
programming  problem.  The  algorithms  developed  by  Hillier  [6,5]  and  extended 


f 


by  Ibarakl  et  al  [7]  fall  into  the  latter  category.  One  of  their  distinc- 
tive features  is  that  the  search  for  good  integer  solutions  Is  focused  in 
the  neighborhood  of  the  optimal  solution  for  the  corresponding  linear  pro- 
gramming problem  (i.e.,  the  original  problem  except  for  deleting  integrality, 
restrictions)  obtained  by  the  simplex  method.  Specifically,  the  following 
three-phase  approach  is  used.  Phase  1 identifies  a path  leading  from  the 
optimal  linear  programming  solution  into  the  interior  of  the  feasible  region 
(when  ignoring  integrality  restrictions).  (Hillier  originally  proposed  a 
linear  path  and  Ibarakl  et  al  extended  this  to  a piecewise  linear  path.) 

In  conceptual  terms.  Phase  2 then  moves  slowly  along  this  path,  using  it  as 
a "home  base"  from  which  to  search  for  a nearby  feasible  (integer)  solution. 

Phase  3 attempts  to  move  from  the  feasible  solution  obtained  to  a succession 
of  better  ones.  The  final  solution  obtained  is  the  desired  approximate  solution. 

One  device  for  attempting  to  improve  further  the  quality  of  the  approxi- 
mate solution  is  to  generate  a number  of  distinct  final  solutions.  The  most 
effective  means  that  Hillier  [6,5]  found  to  accomplish  this  is  to  repeat  Phase  2 
and  3 by  moving  down  the  path  in  Phase  2 beyond  the  point  at  which  the  previous 
Phase  2 solution  had  been  found.  He  also  found  evidence  to  indicate  that  the 
location  of  the  Phase  2 solution  relative  to  the  constraints  may  have  a strong 
influence  on  the  quality  of  the  solution  determined  by  Phase  3.  That  is,  the 
degree  of  maneuverability  allowed  by  the  Phase  2 solution  seems  to  be  an  impor- 
tant consideration.  These  two  factors,  substantial  movement  along  the  Phase  1 
path  and  degree  of  maneuverability,  indicate  that  the  choice  of  a "home  base" 
path  to  follow  into  the  feasible  region  in  Phase  2 deserves  careful  attention. 

The  selection  of  this  path  is  the  subject  of  this  paper. 
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2. 


Notation 


The  problem  under  consideration  is  to  choose  x , . . . so  as  to 


(1) 


maximize  Z = I c.x,, 

j=l  J J 


subject  to 


(2) 


t a . . X , < b . , 

• 1 ij  J ■"  1 

J=1 


i » 1,2, ... ,m 


(3) 


X.  > 0, 
J - 


j - 1,2, ... ,n 


(4) 


Xj  integer 


j = 1,2, . . . ,n' , 


sox  ...,x  need  not  be  an  integer.  Let  x be  the  n-vector  (x, ,x„,...,x  ), 
n +i  n — J.  / n 

and  let  be  the  optimal  linear  programming  solution  obtained  by  the  simplex 

method  by  neglecting  the  integer  constraint  (A).  Define 

O) 


(5) 


B “ (j  I j ^ n'  and  Xj  is  a basic  variable). 


and  let  N be  the  number  of  elements  of  B.  Given  any  x,  let  Xg  denote  the  cor- 
responding N-vector  whose  components  are  x^  and  that  j c B.  Finally,  let 

n 

d.(x)  be  the  Euclidean  distance  from  x to  the  hyperplane  I a^ .x.  = b.,  and 

J = 1 

when  projecting  from  x-space  to  x„-space,  let  d^(x„)  be  the  Euclidean  distance 

— — D 1 — B 

from  x_  to  the  hyperplane  I ~ ^ ax.  Thus  (after  introducing  a 

jeB  J ^ j^B  J 

sign  convention), 

(6) 


n "22 

d (x)  - (b  - Z a X )/(  Z a ) , 


1 = 1,2, .. .m 


(7) 


d.(x  ) “ (b  - Z a X )/(  Za,,) 
i B i ij  j ij 


1,2, 


3 


3. 


Review  of  Previous  Methods 


Hillier  [6,5]  originally  proposed  and  tested  two  methods  for  choosing 
the  interior  path  in  Phase  1.  Both  methods  are  based  on  the  following  general 
approach.  First,  use  the  simplex  method  to  obtain  Then,  for  each  func- 
tional constraint  1 In  (2)  that  is  binding  at  (l.e.,  whose  slack  variable 

is  nonbaslc  for  replace  b^  by 

(8)  bj^^  “ *’i  " ^1 

for  some  suitably  chosen  scalar  ^ 0.  For  this  modification  of  the  linear 
programming  problem  (1-3),  use  standard  post-optlmallty  calculations  to  obtain 
the  new  solution  having  the  same  basis  as  Note  that  x^^^  also  is 

feasible  for  the  original  linear  programming  problem  (1-3)  except  perhaps  for 
some  of  the  constraints  that  are  not  binding  at  x^^^*  Therefore,  the  objective 
of  Phase  1 is  fulfilled  by  letting  the  line  segment  between  x^^^  ®*nd  x^^^  be 
the  desired  interior  path. 

Method  1 sets 


so  that  every  basic  variable  that  is  restricted  to  be  integer  by  (4)  can  be 
changed  by  as  much  as  ^ (as  when  rounding  to  the  nearest  integer  value)  with- 
out violating  the  original  constraint  1 in  (2) . 

Method  2 sets 

1 1 

1^222 
(10)  A.  - ^(  I a ) N , 

J-1 


so  that 


(11)  d^(x^^^)  - -|n^. 


r 


These  methods  are  Illustrated  In  Figure  1 for  a problem  with  functional  con- 
straints ^ 5 and  + 2x2  ^ 5,  where  the  objective  function  is  such  that 
both  of  these  constraints  are  binding  at  x^^\ 


such  that  j L B 


Both  methods  have  the  key  property  that  every  Xj  " 

can  be  rounded  to  the  nearest  integer  value  wltiiout  violating  any  of  the 

original  constraints  (2-3)  that  are  binding  at  (This  holds  for  Method 

(2) 

2 since  the  Euclidean  distance  between  this  rounded  solution  and  x is  at 


most  — N .)  Therefore,  since  Phase  2 begins  searching  for  feasible  solutions 

by  rounding  solutions  lying  on  the  Phase  1 Interior  path,  this  property 

guarantees  that  a rounded  solution  satisfying  at  least  the  binding  constraints 

(2) 

will  be  obtained  by  the  time  is  reached  (and  usually  much  sooner) . If 

some  such  solution  that  also  satisfies  the  nonbinding  constraints  can  be 
found,  it  will  provide  the  desired  starting  feasible  solution  for  Phase  3. 

Broadly  speaking,  the  basic  objective  in  choosing  an  interior  path 
i ive  it  lead  into  the  "middle"  of  the  feasible  region  where  it  should 

t to  locate  a good  feasible  solution,  and  where  the  located  feasible 
.o^utions  should  provide  the  greatest  latitude  for  finding  improved  solutions 
in  Phase  3.  Both  methods  appear  to  meet  this  objective  quite  well.  In  fact, 
it  follows  from  (11)  that  the  Method  2 line  segment  is  equidistant  from  the 
functional  constraint  hyperplanes  that  are  binding  at  x^^^ . Method  1 does 
not  have  this  property,  but  it  does  consider  the  additional  relevant  question 
of  just  how  far  rounding  a solution  can  move  it  toward  a given  hyperplane. 
Thus,  the  Method  1 path  can  be  thought  of  as  equidistant  from  the  binding 
hyperplanes  when  the  distance  is  measured  in  the  direction  that  the  maximum 
possible  amount  of  rounding  can  move  a point  toward  the  given  hyperplane. 

The  next  section  evaluates  more  critically  just  how  well  these  methods 


actually  meet  this  objective  when  it  1s  defined  more  carefully. 
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^ • Evaluation  of  Previous  Methods 

The  two  previous  methods  discussed  In  Section  1 botli  locus  on  extreme 
cases.  What  is  the  largest  possible  distance  that  rounding  can  move  a solu- 
tion? (Method  2)  Wliat  Is  the  largest  possible  Increase  in  the  left-hand 
side  of  a functional  constraint  that  can  result  from  rcunding.^  (Method  1) 
This  raises  the  question  of  just  how  these  extreme  cases  compare 
with  the  actual  cases  that  normally  would  be  occurring  on  a statistical 
basis.  This  is  evaluated  below,  first  in  regard  to  the  distances  moved  by 
rounding,  and  then  the  changes  in  left-hand  sides.  Implications  are  then 
explored . 

To  study  this  question  statistically,  assume  that  the  parameters  of 
the  problem  (1-4)  are  drawn  from  some  underlying  probability  distributions. 
Assume,  without  loss  of  generality,  that  the  ordering  of  the  variables  is 
such  that  B = {1,2,. ..,N}.  Then,  for  some  given  method  of  calculating  the 

(9) 

needed  to  determine  x “ (see  Section  3),  let  the  random  variable  X.  be 

(2) 

the  change  in  resulting  from  rounding  it  to  the  nearest  integer,  for 

j = 1,2,...,N.  Under  most  circumstances,  one  would  expect  the  fractional 

(2) 

parts  of  the  x^  to  be  dispersed  rather  uniformly  over  the  unit  interval. 
(For  certain  highly  structured  problems  with  integer-valued  parameters,  the 
X.j  may  have  discrete  distributions  with  considerable  mass  at  zero.  Such 
problems  tend  to  be  relatively  easy  to  solve,  however,  and  so  are  of  less 
interest  for  heuristic  procedures.)  Therefore,  it  is  assumed  here  that 
"^1’  ‘2’  independent  and  identically  distributed  according  to  a 

uniform  distribution  ranging  from  -y  to  +■!■.  so  that  this  common  distribu- 
tion has  the  density  function 


(12) 


f(x) 


( 

othei-wlse. 


1,  for  1 X < -| 


(2) 


Now  let  the  random  variable  D be  the  Eaclidean  distance  that  x 

(2) 

moves  due  to  rounding  Xj  to  the  nearest  integer  for  j = so 

that 


N 


1 


(13) 


D = ( Z X^)^. 
3=1  J 


Theorem  1:  As  N grows  large,  the  probability  distribution  of  D tends  to 

a normal  distribution  with  mean  and  variance, 

_1 

(U)  E(D)  = (~■^^  VanD)  = — . 


Proof : 


Let  the  random  variable  Z have  a normal  distribution  with  mean 


zero  and  variance  one,  and  let 


(15) 


V i i 

^ 2 N 2 

( 1 Xj)^  - (^)^ 

, for  N = 1,2, . . . . 

(— )^ 


Thus,  letting  -*•  denote  convergence  of  a sequence  of  random  variables  in  the 
sense  of  poinCwise  convergence  of  their  cumulative  distribution  functions, 
it  is  sufficient  to  show  that 


It  is  easily  calculated  that 


(16) 


E(X^)  - i , Va.  i 


I 


i 

[ 


Therefore,  by  t.he  Central  Limit  Theorem, 


D‘  - 

--P  . Z. 


'■180'' 


Now  construct  a Taylor  series  expansion  of  the  square  root  of  D 


2 N 

about  its  mean,  E(D  ) = yj  » 


I 


(17) 


^ ,N  2 ^ 1,N.2,,2  N,  1 , 2.,,2  N,2 

D + 2(^2^  “ 12^  ~ 8 ^ "12^ 


2 N 

for  some  ^ interior  to  the  interval  joining  D and  (so  ^ is  a function 


2, 


of  the  value  taken  on  by  D ).  Consequently, 


(18) 


S = 


0 - (JL)2  jj2  _ J_ 

'■12''  12 


1 

.1.^2 

'60^ 


(-^) 
V 1 or\' 


'a’ 


^180' 


where 


'n  ' 8^^°^  ^180^^ 


12 


(-^-) 

'■180'^ 


SO 


1440(60) 
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Now  note  that 


since 


implies 


n-H' 


J_(JL)2 


1 + 


^ 12I 


JL 

12 


with  probability  one, 


C-- 
^ 12 

'■180-' 

|z| 


Z 12 

_N_ 

12 


0 with  probability  one. 


Therefore, 


0 

N 


with  probability  one. 


Q.E.D. 


Although  Theorem  1 gives  asymptotic  results,  these  also  can  be  used 
as  approximations  for  even  small  values  of  N. 

The  Central  Limit  Theorem  usually  will  give  a reasonably  good  approximation 
to  the  normal  distribution  for  values  of  N as  small  as  four  or  five.  Fur- 
thermore, an  analysis  of  additional  terms  in  the  above  Taylor  series 
expansion  (17)  indicates  that  the  expresssions  for  E(D)  and  Var(D)  given  by 
Theorem  1 should  be  within  a few  percentage  points  of  the  true  values  for 
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values  of  N this  small.  This  implies  that  Var(D)  is  essentially  a constant 
independent  of  N,  even  though  E(D)  essentially  is  growing  proportionally 

to  N^. 

It  is  interesting  to  compare  the  range  of  possible  values  of  D, 

I 

(19)  0 1 D 1 -|n^ 

with  the  range  of  likely  values  suggested  by  Theorem  1.  For  example,  using 
the  mean  + nearly  four  standard  deviations  gives 


(20) 


Prob  { (~)^  - 1 D 1 2 ^ " 0.9998. 


Thus 


1 

1..2 


, even  though  Method  2 focuses  on  the  upper  bound,  D 5.  N > for  all 


practical  purposes  one  can  instead  assume  that  D ^ (N/12)  + -j.  Furthermore, 

for  fairly  large  values  of  N,  the  actual  values  taken  on  by  D will  tend  to 

2 

be  quite  close  to  3 ~ 58%  times  the  upper  bound  used  by  Method  2. 


1 

1 2 

Even  more  pertinently.  Method  2 uses  N as  an  upper  bound  on  the 
distance  that  rounding  a solution  can  move  it  toward  a given  functional 
constraint  hyperplane,  and  Method  1 uses  a similar  quantity.  By  contrast, 

D represents  the  distance  moved  in  whichever  direction  results  from  the 
rounding,  so  the  resulting  decrease  in  the  distance  to  the  hyperplane  is 
likely  to  be  much  smaller  than  D.  Therefore,  a realistic  statistical  upper 
bound  on  this  decrease  may  be  far  smaller  than  the  upper  bounds  used  by 
these  methods. 
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To  study  this  question,  let  the  random  variable  and 

(1  = l,2,...,m)  be  the  decrease  in  d^(x)  and  dj^(Xg),  respectively,  due  to 
(2) 

first  setting  x “ 2L  rounding  Xj  to  the  nearest  integer  for 

j = 1,2, ...,N.  Thus 


£ a,  .X, 


E a,  .X. 

1-1  ‘J  ' 


2 2 


“^2  2 
( r a 

j=l 


Df  = CDj^,  where  C = 


n 2 
Z af. 

J=1 


, 0 < C < 1. 


Theorem  2: 


(a)  E(D^)  = 0, 


E(Dj^) 


Var(D^)  = j2  » 


Var(Dp 


Prob  {D  < k— }>  1 ^ , 

Vl2  - 


Prob  {D'  < k — } > 1 — ^ 

Vl2  " k^ 


for  any  k > 1. 


I 1 

l,b)  AS  N grovs  large.  It  necessarily  is  a uniformly 

bounded  sequence  such  tliat  its  sum  diverges,  then  c'tie  probability  distribution 

of  U.  and  D'  tends  to  a normal  distribution  with  Che  respective  means  and 
1 i 

variances  given  above. 

Proof : P.;rt  ta)  foilows  immediately  by  direct  calculation  of  the 

moments  and  tl)en  Chebychev's  inequality.  Part  (b)  is  a direct  application 

1 

of  the  Lind' oerg  ..ondition  for  the  Central  Limit  Theorem. 

The  condition  on  the  parameters  in  Theorem  2h  is  not  a particularly 
restrictive  one.  For  example,  LI.  holds  if  these  coefficients  for  a given 
constraint  can  be  thougiit  of  as  being  Independently  drawn  from  a single  under- 
lying probability  distribution  that  is  bounded  but  not  degenerate  at  zero. 

However,  the  rate  at  which  the  distribution  of  and  D’  approaches  a normal 

2 

distribution  depends  considerably  on  the  dispersion  of  the  If  ^ very 

few  coefficients  dominate  the  others  in  magnitude,  then  the  normal  approxima- 
tion will  not  yet  be  a good  one.  But  if  there  are  many  relatively  large 
coefficients  of  the  same  order  of  magnitude,  it  will  be  an  excellent  approxi- 
mation. 

If  N is  large  and  the  normal  approx-i matlon  is  reasonable,  then  the 
statistical  upper  bounds  become  vt’ry  small  Indeed  compared  to  the  bounds 
used  by  Methods  1 and  2.  For  example,  using  the  mean  plus  'VTT  standard 
deviations , 


(23) 


ProblD^  ^ C)  > 0.9997,  ProblD^  1 1}  > 0.9997. 


See  Feller  (3,  pp.  256-258]. 
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Note  that  neither  bound  ever  exceeds  one  regardless  of  how  large  N is. 


By  contrast.  Method  2 uses  the  upper  bound, 

I 

(24) 


and  Method  1 uses 


(25) 


1 


N 


( a^.) 

:=i 


1 • 

2 


(Observe  that  (24)  and  (25)  give  the  same  upper  bound  if  N = n and  the  a^^ 

are  equal,  but  that  (24)  is  larger  otherwise.)  Thus,  these  latter  bounds 

can  be  many  times  too  large  to  reflect  realistic  possibility,  and  so  lose 

their  relevance  for  their  intended  purpose. 

The  implication  is  t.hat  each  in  (8)  can  sometimes  be  made  much 

(2) 

smaller  in  order  to  obtain  an  appropriate  x , e.g.,  by  using  (23)  to  set 

1 M 1 

*^2  2 “22 

(26)  A,  = C(  1 a^,)^  = ( E a..)  . 

j=l  j=l 

(2) 

Then,  if  one  wants  to  use  x directly  to  obtain  a feasible  solution  by 
rounding  (bypassing  Phase  2),  a far  superior  solution  may  result.  Further- 
more, this  solution  is  far  less  likely  to  violate  any  of  the  constraints 
(2,3)  that  are  not  binding  at  x^^^  than  are  the  rounded  solutions  resulting 
from  the  larger  A^  in  (9)  and  (10). 


l4 


However,  if  the  fuiJ  three-phase  procedure  is  to  be  used,  then  the 

real  significance  of  is  not  the  resulting  but  rather  the  direction 

of  the  line  segtnent  leading  from  toward  That  Is,  two  methods 

for  setting  the  A^  that  differ  by  only  a fixed  multiplicative  constant  (for 

a given  problem)  actually  would  be  equivalent  in  this  respect  since  the 

direction  would  be  the  same.  This  is  the  case,  for  example,  for  the  A^ 

in  (10)  and  the  A^  suggested  by  the  statistical  upper  bound  in  (20).  However, 

note  that  the  A^  In  (2A)  are  not  equivalent  to  the  A^  in  either  (9)  or 

(10),  so  (26)  does  give  a new  direction  for  the  line  segment. 

It  is  enlighteiung  to  reflect  on  the  differences  between  (26)  and 

(10).  At  first  glance,  they  appear  to  be  quite  similar  since,  except  for 

(10)  having  the  multiplicative  constant  given  in  (11) , they  differ  only  in 

the  limit  of  summation  being  n for  (10)  and  N for  (26).  However,  this  minor 

symbolic  difference  masks  two  major  methodological  dl f f eren(PR . First,  (10) 

gives  as  much  influence  to  the  integer-restricted  variables  (4)  that  arc 

nonbasic  at  (x,, ,,, x^, ,x  ,)  as  to  those  that  are  basic  (x,  ,x, , . . . ,x„)  , 

\+]  n i Z N 

even  though  these  nonbasic  variables  are  Irrelevant  for  rounding  solutions 

between  (and  play  little  role  thereafter  in  Phase  2).  Second, 

(10)  also  gives  as  much  influence  to  the  variables  that  are  not  integer- 

restricted  (x  ,..,x  ...,x  ),  even  thougli  these  variables  play  only  a 

n -^1  n +2  n ° 

secondary  roie  at  best  in  any  versions  of  Phase  2 developed  to  date.  The 
search  for  a good  feasible  solution  in  Phase  2 takes  place  primarily  in 
space  (using  the  orthogonal  projection  onto  this  coordinate  hyperplane  from 
the  parallel  hyperplane  passing  through  the  current  point  of  interest  on  the 
line  segment  between  Therefore,  these  differences  suggest 

some  major  drawbacks  in  Method  2,  which  Is  based  on  (10). 


15 


To  illustrate  these  drawbacks,  suppose  ihat  ii  = 3 for  the  problem 


with  N = 2 siiown  in  Figure  1,  where  the  additional  variable  enters  only 

into  the  4x,  < 5 constraint  as  Ax,  + A0x„  < 5.  This  changes  x„^^  for 
i — l i — -D 

Method  2 to  Xq"^  = H202,  7-  + 7 */202)  = (-5.86,7.36),  so  that  the  line 

— 15  H Z 4 4 

(1)  9 

segment  from  Xg  now  intersects  the  X2~axis  s llgiit ly  above  X2  = The 

result  is  that  the  essentially  irrelevant  variable  x^  has  greatly  distorted 

the  direction  of  the  Phase  2 search  for  a good  feasible  solution  by  moving 

it  far  away  from  the  center  of  the  feasible  region  in  5Cg-space. 

Therefore,  it  appears  to  be  more  appropriate  to  focus  on  distances 

in  X -space,  such  as  the  Dl  variables  considered  above,  rather  than  distances 

in  x-space.  This  conclusion  leads  to  the  new  method  presented  in  the  next 

section. 


5 . A New  Method 

(2) 

Based  on  the  above  analysis,  it  is  proposed  that  x be  obtained  by 
making  it  equidistant  in  x„-space  from  the  functional  constraint  hyperplanes 

— D 

that  are  binding  at  • Thus,  the  rationale  is  the  same  as  for  Method  2, 

except  that  the  d.(x„)  quantities  in  (7)  now  are  equated  rather  than  the 
dj^(x)  in  (6).  To  be  completely  analogous  to  (10)  and  (11)  for  Method  2,  one 
would  set 

(27)  A - 

TtB 


I 


so  that 


^ (2),  1„2 

di(xB  ) = 2^ 


This  would  guarantee  that  can  be  rounded  to  the  nearest  integer  for 

every  j e li  and  the  resulting  solution  would  still  satisfy  all  of  the  con- 
stralnts  that  are  binding  at  • However,  (2J)  indicates  that  (28)  often 
is  overly  conservative,  since 

(29)  ^ 


can  virtually  provide  the  same  guarantee  (assuming  the  conditions  of  Theorem 
2b  hold).  As  already  observed  in  Section  4,  this  statistical  upper  bound 
on  the  leads  to 


jeB 


With  either  (27)  or  (26),  2i''  still  would  be  calculated  by  the  procedure 

described  at  the  beginning  of  Section  3. 

Note  that  (26)  and  (27)  actually  provide  equivalent  methods  (hereafter 

labeled  Method  3)  since,  for  any  given  problem,  they  differ  only  by  the 

1 

1 2 

fixed  multiplicative  constant  -jN  and  so  give  the  same  direction  for  the 
line  segment  from  to  x^^^.  Nevertheless,  when  Theorem  2b  is  applicable, 

it  is  suggested  that  (26)  be  used,  since  its  provides  a more  realistic 

estimate  of  the  maximum  amount  by  which  the  line  segment  should  need  to  be 
extended  from  • In  the  rare  instances  where  rounding  this  x^^^for  j e B) 
violates  a binding  constraint,  the  line  segment  always  can  be  extended  a 


little  further.  However,  when  this  rounded  solution  is  not  feasible,  the 


usual  explanation  would  be  that  it  violates  one  or  more  of  the  constraints 
that  are  not  binding  at  would  do  no  good  to  extend  the  search 

further  in  the  direction  of  the  line  segment.  Therefore,  failure  to  find 
a feasible  solution  by  this  time  would  be  a good  signpost  that  the  search 
needs  to  be  moved  in  another  direction  to  take  into  account  these  relevant 
nonbinding  constraints.  A procedure  for  doing  this  is  described  in  the 
next  section. 

Another  reason  for  preferring  (26)  over  (27)  is  its  strong  superiority 

for  the  following  new  option  for  streamlining  Phase  2.  Begin  (as  usual)  by 

checking  whether  rounding  x^^^  (for  j e B)  provides  a feasible  solution. 

(2) 

If  not,  then  immediately  check  whether  rounding  x (for  j e B)  provides 

a feasible  solution.  Only  if  this  fails  also  would  Phase  2 be  resumed  in 

the  usual  way.  (Another  possibility  would  be  to  conduct  the  usual  Phase  2 

search  from  both  of  these  rounded  solutions  before  considering  any  other 

points  on  the  line  segment  between  x^^^  and  The  rationale  is  that 

if  rounding  x^^^  (and  perhaps  searching  from  there)  fails,  then  Identifying 

a nearby  feasible  solution  may  be  difficult,  leading  to  a long  Phase  2 

(2) 

process.  By  using  the  "tight”  x provided  by  (26),  much  time  might  be 
saved  by  skipping  down  to  where  a feasible  solution  may  be  readily  identi- 
fied, while  perhaps  sacrificing  little  in  the  quality  of  this  solution  (or 
of  the  resulting  final  solution  yielded  by  Phase  3). 
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6 . Extension  of  Methods  to  Piecewise  Linear  Interior  Paths 

All  of  the  methods  discussed  above  focus  on  the  functional  constraints 

(2)  and  the  nonnegatlvltv  constraints  (3)  that  are  binding  at  (i.e., 

such  that  the  slack,  variable  In  (2i)  or  the  variable  in  (3j)  is  nonbaslc 

for  requiring  that  rounding  (for  j e B)  should  satisfy  these 

(2) 

constraints.  There  is  no  requirement,  however,  that  either  x or  the 

corresponding  rounded  solution  must  (with  even  high  probability)  satisfy 

all  of  the  constraints  (2,3)  that  are  not  binding  at  If  rounded 

solution  does  violate  any  of  these  constraints,  then  It  becomes  possible 

(but  by  no  means  certain)  that  Phase  2 will  fall  to  find  a feasible  solution 

in  the  general  region  between  and  Furthermore,  if  any  of  the 

variables  among  }c  or  the  slack  variables  added  to  (2)  are  degenerate  basic 

variables  at  » the  line  segment  from  x*'^^  fo  immediately 

out  of  the  feasible  region.  Although  even  this  may  not  prevent  finding  a 

feasible  solution,  either  complication  would  likely  blunt  the  effectiveness 

of  the  otherwise  highly  successful(optional)  technique  proposed  by  Hillier 

[6,5]  of  generating  multiple  final  solutions  in  Phase  3 by  generating  a 

series  of  feasible  solutions  in  which  each  succeeding  feasible  solution 

corresponds  to  a new  point  further  along  the  line  segment  from 

(2) 

X . These  considerations  suggest  that  the  choice  of  an  interior  path  to 
follow  during  Phase  2 preferably  should  be  governed  by  the  location  of  the 
path  relative  to  ail  constraints  of  the  problem,  not  only  those  which  are 
binding  at  x^^^ • 


19 


An  attractive  alternative  choice  of  this  kind  for  the  interior  path 
is  the  parametric  solution  to  the  linear  program, 

(30)  maximize  r, 

subject  to 

n 

(31)  Zax.+Ar^b.,  1=1,.. .,m 

j=l  J ^ ^ 

n 

(32)  r c X.  = Z 

j=i  j J 

(33)  Xj  i 0»  j = 1, . . . ,n 

(34)  r > 0, 

as  Z is  decreased  from  its  value  at  . Thus  this  extension  of  Methods 

1,  2,  and  3 (labeled  IP,  2P,  and  3P,  where  P stands  for  "Parametric")  would 
use  their  respective  definitions  of  A^,  namely,  (9),  (10),  and  (26) (or  (27)). 
Because  of  the  properties  of  parametric  linear  programming,  the  resulting 
path  for  each  method  is  continuous  and  piecewise  linear,  leading  from  x^^^ 
into  the  interior  of  the  feasible  region  for  the  continuous  version  of  the 
original  problem  (1-3). 

Since  there  no  longer  is  a fixed  set  of  variables  (corresponding  to 
B)  that  need  rounding  along  such  a path,  one  could  adjust  the  A^  each  time 
this  set  changes  by  redefining  B and  N in  terms  of  the  current  basts  instead 
of  This  is  not  recommended,  since  it  would  considerably  complicate 
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the  computations,  Introduce  discontinuities  into  the  path,  and  perhaps 
create  chain  reactions  of  changes.  However,  the  price  being  paid  for 
fixing  the  should  be  recognized.  Since  the  current  values  of  such  re- 
defined B and  N may  indeed  change  with  each  new  basis  in  the  parametric 
solution,  the  original  rationale  for  choosing  the  fixed  may  no  longer 
be  completely  applicable.  Fortunately,  this  doesn't  really  affect  Method 

2P,  since  its  A^  (10)  is  uninfluenced  by  B and  N except  for  the  same 

1 

2 

multiplicative  factor  N for  all  1,  which  only  changes  the  scale  of  r. 

However,  B does  play  significant  role  in  (9)  (Method  IP)  and  in  (26) 

(Method  3P) . The  relative  amount  by  which  changes  in  B would  change  these 

A^  depends  greatly  on  the  size  of  N and  the  dispersion  of  the  With 

N large,  and  with  a^^  of  comparable  magnitude  for  different  j e B,  any 

deviations  between  the  A.  for  the  "current"  B and  the  fixed  A.  at  x^^^ 

1 i — 

should  be  of  little  consequence- 

If  N is  not  large,  or  if  a few  a^^  dominate  the  others  in  magnitude, 
then  it  is  recommended  that  the  integer-restricted  variables  (4)  that  are 
nonbasic  at  x^^^  fixed  at  zero  in  (30-34)  for  Methods  IP  and  3P . This 
would  ensure  that  the  original  rationale  for  (9)  and  (26)  would  continue  to 
hold  (conservatively)  along  the  entire  path,  since  it  would  restrict  the 
rounding  of  x to  just  x . The  drawback  is  that  it  decreases  the  flexibility 

— D 

in  choosing  an  appropriate  interior  path  in  jc-space. 

It  should  be  noted  that  Ibaraki  et  al  [7]  first  proposed  this  approach 
for  generating  a piecewise  linear  Interior  path  in  the  context  of  Method  2. 
(However,  they  also  acknowledge  [7,p.l34]  the  independent  development  of  this 
approach  by  the  present  authors.) 
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The  piecewise  linear  Interior  path  generated  by  each  ol  these  tliree 
methods  follows  a "most  interior"  route  into  the  feasible  region  for  (1-3) 


in  the  sense  that,  for  every  value  of  Z,  the  corresponding  point  along  the 
route  maximizes  the  minimum  Euclidean  distance  to  the  boundary  hyperplanes 
for  the  functional  constraints  (2),  when  this  distance  is  measured  in  the 
appropriate  way.  For  Method  IP,  the  distance  to  the  hyperplane  for  constraint 
i should  be  measured  from  the  point  in  the  direction  of  the  n-vector  ^ such 
that  d.  = +1  if  j t B and  a..>0,  d.  =-lifjcB  and  a.,  < 0,  and  d.  = 0 
otherwise.  For  Method  2P,  the  distance  is  measured  in  the  usual  way  per- 
pendicularly to  the  liyperplane,  as  in  (6).  For  Method  3P,  it  is  done  in  the 
same  way  as  for  2P  except  in  j^-space  rather  than  jc-space,  as  given  by  (7). 
Thus,  for  Methods  2P  and  3P,  the  point  along  the  route  for  a given  value  of 
Z is  the  center  of  the  largest  sphere  which  may  be  inscribed  within  the 
functional  constraints  in  the  appropriate  space,  subject  to  the  side  condi- 
tions that  the  center  must  be  nonnegative  and  lie  on  the  hyperplane  (32). 

When  Method  3P  uses  (26),  the  radius  of  the  sphere  in  x^-space  is  the  optimal 

D 

value  of  r defined  by  Z.  This  also  would  be  the  radius  in  5c-space  for  Method 

1 

2P  if  the  unnecessary  multiplicative  factor,  were  eliminated  from  (10). 

The  paths  for  Methods  2P  and  3P  are  also  motivated  by  the  observation 

1 

1 2 

that  every  sphere  in  Euclidean  N-space  of  radius  r ^ y N contains  at  least 

one  lattice  point,  namelv,  the  point  which  results  from  rounding  the  center 
of  the  sphere.  (N  here  should  be  interpreted  as  the  current  number  of  vari- 
ables that  need  rounding.) 


Suppose  r*  is  tne  radius  of  the  lai'gest  sphere  with  nonnegative 

center  which  may  be  Inscribed  within  the  functional  constraints  In  jt-space. 

The  center  of  the  sphere  x*  lies  along  the  Method  2P  interior  path.  Fur- 

1 

1 2 

theriaore,  if  r*  • the  heuristic  procedure  is  guaranteed  to  find 

a feasible  solution  by  proceeding  along  this  interior  path,  since  at  worst, 

it  will  round  x*  to  find  a feasible  integer  solution.  These  same  remarks 

also  apply  to  Method  3P  in  x -space,  with  n'  replaced  by  N (as  defined  in 

o 

Section  2),  if  the  Integer- restriced  variables  (4)  that  are  nonbasic  at 

have  been  fixed  at  zero  in  (30-34).  Furthermore,  when  the  conditions 
of  Theorem  2b  hold,  (23)  indicates  tnat  r*  ^ C (for  Method  2P)  or  r*  ^ 1 
(for  Method  3P)  usually  would  be  sufficient  to  find  a feasible  solution. 

Another  interesting  property  of  the  path  generated  by  Method 
iP(i  = 1,2,3)  is  that,  barring  degeneracy,  its  first  segment  leading  away 
from  lies  along  the  Method  i line  segment.  The  straightforward  proof 

of  this  statement  follows  from  the  correspondence  between  the  optimal  basis 
to  problem  (1-3)  and  the  initial  basis  in  the  parametric  problem  (30-34). 
This  initial  basis  to  (30-34)  will  have  as  basic  variables  r and  all 
variables  basic  at  As  Z is  decreased,  this  basis  remains  optimal 

until  one  of  the  basic  variables  is  driven  to  zero  at  some  critical  value 
of  Z.  At  this  point  the  basis  changes  and  the  interior  path  diverges 
from  the  corresponding  Method  1 line  segment. 
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Another  Extension  to  Piecewise  Linear  interior  Patlis 

The  above  approach  to  generating  piecewise  linear  interior  paths 
does  require  a software  package  that  includes  parametric  programming,  as 
well  as  the  considerable  execution  time  involved.  Tf  this  is  not  convenient, 
then  the  following  is  a simpler  and  quicker  approacii  that  only  requires  the 
basic  simplex  method  needed  to  obtain  anyway. 

First,  apply  the  simplex  method  to  solve  the  following  variation  of 

(1-3), 


(35)  maximize  r. 


subject  to 
(36) 


E a.  X.  + Ar  < b, , 
j-l  O J i - 1 


m 


(37)  Xj  >0,  j = 1,2, 

starting  with  the  initial  solution  ‘ Record  the  sequence  of  basic 

feasible  solutions  generated  in  the  process  of  doing  this,  and  then  connect 
each  successive  pair  of  these  solutions  (in  2c-space)  by  a line  segment.  This 
is  the  desired  piecewise  linear  path. 

As  before,  this  extension  of  Methods  1,  2,  and  3 (labeled  IS,  2S,  and 
3S,  where  S stands  for  "Simplex")  would  use  their  respective  definitions  of 
A^,  namely,  (9),  (10),  and  (26)  (or  (27)). 

Although  the  path  generated  by  Method  IS  (i  = 1,2,3)  leads  to  tlie  same 
"most  interior"  point  (according  to  the  respective  distance  norms  described 
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In  Section  6)  as  tor  Kethod  il’,  It  di>es  not  follow  a "most  Interior"  route 


in  the  same  sense.  However,  it  does  provide  an  alternate  route  to  this 
"most  interior"  point  which  never  leaves  the  feasible  region  for  (1-3) . 


8.  Computational  Experience 


In  order  to  evaluate  and  compare  these  techniques,  several  FORTRAN 
codes  were  developed  for  a CDC-b400  system.  Since  Phase  1 of  the  heuristic 
algorithm  developed  by  Ibaraki  et  al  [7]  is  based  on  Method  2 (and  since 
programming  services  were  no  longer  available  when  Method  3 v;as  developed) , 
the  focus  was  on  testing  variations  of  Method  2 (2,  2P,  and  2S),  although 
Method  1 (without  extension)  also  was  run  on  the  problems. 

A total  of  64  test  problems  were  used.  22  of  these  are  standard 
test  problems  in  the  literature  - Haldi's  IBM  problems  (#1-5,9)  and  Fixed 
Charge  problems  (all  10),  and  Woolsey's  4-point  and  5-polnt  combinatorial 
problems  (all  18  are  reproduced  by  Trauth  and  Koolsey  [15]),  plus  two 
problems  (#4,5)  given  by  Petersen  [9]  and  two  (#8,9)  given  by  Thompson  [13]. 
The  other  42  are  randomly  generated  problems  that  were  previously  used  by 
the  authors.  These  consist  of  the  Type  I problems  (#1-8,  and  "Large" 


30  X 60,  60  X 50,  60  x 60),  Type  II  problems  (#1-16),  and  Type  III  problems 
(#2-5,6)  discussed  in  [6],  as  well  as  the  Type  V problems  (#1-10)  discussed 


in  [2].  Of  particular  interest  are  the  type  I and  II  problems,  whose  a^^ 
parameters  are  randomly  generated  integers  from  the  intervals,  [-40,59]]  and 


[0,99],  respectively,  and  whose  variables  are  general  integer  variables  (as 


opposed  to  binary  variables).  All  64  problems  were  treated  as  pure  integer 
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programs,  so  n'  = n. 

Since  the  relevant  consideration  in  evaluating  Phase  1 methods  Is 
the  resulting  effectiveness  of  the  overall  heuristic  procedure,  these  methods 
were  applied  to  the  test  problems  in  conjunction  with  fixed  Phases  2 and  3. 

These  latter  phases  used  Methods  3A  and  i respectively,  according  to  the 

labels  described  in  [6].  (The  one  minor  exception  is  that  Method  2A  for 

Phase  2 was  used  in  conjunction  with  Method  1 of  Phase  1.)  Phase  2 also 

used  the  device  discussed  in  both  [6]  and  [5]  (and  labeled  R3A  in  [5])  of 
generating  multiple  feasible  solutions  for  Phase  3.  (This  involves  re- 
peating the  Phase  2 search  all  along  the  interior  path  generated  by  Phase 
1,  and  so  provides  a better  test  of  this  path.)  A maximum  of  20  feasible 
solutions  (excluding  any  obtained  directly  from  x^^^)  were  allowed  from 
Phase  2,  where  these  solutions  need  not  be  distinct.  The  algorithm  pro- 
ceeded to  Phase  3 with  each  new  Phase  2 solution  only  if  that  solution 
differs  from  the  immediately  preceding  ones  and  this  was  allowed  at  most  10 
times . 

Procedures  2-R3A-1,  2P-R3A-1,  and  1-R2A-1  were  applied  to  all  64  test 
problems.  For  47  of  these  problems,  they  all  found  their  best  solttion  from 
the  Phase  2 feasible  solution  generated  directly  from  x^^^  so  that  the 
interior  path  generated  by  Phase  1 was  irrelevant  in  these  cases.  For  the 
other  17  Lest  problems,  the  results  regarding  the  best  final  solution  (Including 
the  point  ir>  the  algorithm  at  which  this  solution  was  found)  are  shown  in 
Table  I for  Procedures  2-R3A-1  and  2P-R3A-1.  (Procedure  1-R2A-1  is  excluded 
from  the  table  since  a detailed  comparison  with  Procedure  2-R2A-1  already  is 
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given  in  Table  111  of  [6]  for  most  of  these  same  problems.)  Procedure 
2S-R3A-1  was  applied  to  7 of  these  17  problems,  as  also  shown  in  Table  1. 

Ibaraki  et  al  (7]  also  applied  their  heuristic  algorithm  to  9 of 
these  64  test  problems,  with  the  comparative  results  shown  in  Table  II 
(Thompson  ■ T,  Fixed  Charge  = FC) . Times  are  given  for  applying  the 
simplex  method  to  find  (labeled  LP) , and  then  for  the  total  of  this 

and  all  three  phases  of  the  heuristic  procedure  per  se.  However,  it 
should  be  noted  that  they  used  a different  computer,  a FACOM  230/60, 
which  they  indicate  corresponds  very  roughly  to  the  IBM  360/65  and  the 
UNIVAC  1108.  Furthermore,  although  they  also  generated  multiple  solutions 
on  these  problems  (except  for  FC-7  and  FC-10) , they  only  allowed  four 
distinct  feasible  solutions  from  Phase  2 and  only  used  the  two  best  of 
these  for  Phase  3. 
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9. 


Conclusions 


Under  some  circumstances.  Method  1 and  (particularly)  Method  2 for 

(2) 

Phase  1 can  yield  an  exceedingly  conservative  x . More  seriously,  Method 
2 gives  undue  weight  to  largely  Irrelevant  variables,  and  so  can  distort 
the  resulting  direction  of  the  line  segment  from  x^^^ . Statistical 
analysis  suggests  a promising  new  Method  3 which  avoids  these  shortcomings. 

All  of  these  methods  focus  only  on  the  functional  constraints  that 
are  binding  at  21^^^ • Although  this  usually  should  be  adequate,  there  are 
situations  where  other  constraints  should  influence  the  interior  path. 

The  P (Section  6)  and  S (Section  7)  extensions  to  piecewise  linear  interior 
paths  provide  this  capability.  Furthermore,  the  P extension  starts  off  on 
the  line  segment  for  the  original  method  (barring  degeneracy) , and  so  should 
provide  a comparable  performance  on  ordinary  problems  as  well.  The  S exten- 
sion may  be  more  convenient,  but  Its  other  properties  are  less  certain. 

On  the  64  test  problems,  the  P extension  led  to  a different  best 
solution  from  the  original  method  on  only  four  of  them  (better  on  two, 
slightly  worse  on  two),  so  it  certainly  doesn't  provide  a dramatic  improve- 
ment. However,  it  is  only  designed  to  help  on  those  occasional  problems 
where  additional  constraints  should  be  considered,  without  hurting  on  others, 
and  it  does  seem  to  meet  this  objective.  Furthermore,  it  should  Increase 
the  reliability  of  heuristic  procedures  in  finding  any  feasible  solution  on 
problems  where  this  is  very  difficult.  Since  the  price  for  adding  this  ex- 
tension is  a modest  one.  It  should  prove  to  be  a worthwhile  addition  for  at 
least  full-fledged  production  codes. 
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On  the  other  hand,  the  S extension  often  gives  a different  beat 
solution  (frequently  worse  but  sometimes  better)  from  eitlier  the  P exten- 
sion or  the  original  method.  it  apparently  tends  to  start  off  the  Interior 
path  In  a much  different  direction.  Therefore,  its  main  usefulness  may  be 
as  a supplementary  metliod  to  obtain  additional  final  solutions  to  try  to 
improve  upon  the  original  best  solution  trom  another  method. 

Computational  comparisons  of  the  heuristic  procedures  tested  here 
with  the  one  proposed  by  Ibarakl  et  al  [7]  suggest  that  they  have  similar 
performance  capabilities. 

The  reader  is  referred  to  [5]  for  an  extensive  comparison  of  Methods 
1 and  2 (without  extension),  as  well  as  some  new  options  for  all  three  phases 
and  comprehensive  experimental  results  on  various  versions  of  these  heuristic 
procedures . 
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